feat(debate): optional early-stop on convergence (closes #4)#38
Merged
Conversation
Add an opt-in debate early-stop when answers converge, preserving the historic fixed-rounds behavior exactly when disabled (default). Signal: round-over-round answer stability -- per-member difflib.SequenceMatcher ratio averaged across members usable in both the previous and current round. Deterministic, stdlib-only, offline-testable. A degenerate score (no comparable answers) or any scoring failure degrades to running the remaining fixed rounds; it never crashes the debate. Opt-in is consistent with the cache feature: - config: ConclaveConfig.converge_threshold (float | None, off by default) - library: Council.debate/debate_sync + run_debate converge_threshold param (None defers to config, mirroring cache's None-defers-to-config) - CLI: --converge-threshold FLOAT plus --converge/--no-converge Recorded on CouncilResult.converged + convergence_score (minimal, mirrors how the cache feature added .cached); actual rounds run is len(rounds). converge_threshold is added to the debate cache key (make_key / _cache_key / _cached_run plumbing) so a converged run and a fixed run over identical inputs never collide. Convergence logic lives in modes.py (not council.py). 9 new offline, transport-mocked tests cover: off=full rounds, on+converge=early stop with recorded score, on+never-converge=full rounds, degenerate input fallback, config-defer, the scorer unit, and a cache-key no-collision check (142 -> 151 tests). ruff check + format clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements PDD §9 #2 / issue #4: optional debate early-stop when answers converge, with a configurable convergence signal. Fixed-rounds behavior is preserved exactly when the feature is off (the default).
Convergence signal
Round-over-round answer stability. For each member with a usable answer in both the previous and current round, compute the
difflib.SequenceMatcherratio between its two answer texts (1.0 = identical), then average across those members. High mean = members stopped revising = the debate has stabilized.Why this signal:
difflib) → fully offline-testable, no heavy deps.Opt-in (mirrors the
cachefeature)ConclaveConfig.converge_threshold: float | None(off/Noneby default; malformed/out-of-range values degrade to off with a warning).Council.debate/Council.debate_sync/run_debateconverge_thresholdparam —Nonedefers to config (mirrorscache'sNone-defers-to-config).--converge-threshold FLOAT(implies on) plus--converge/--no-converge(mirrors--cache/--no-cache).--convergewith no explicit threshold uses a conservative default of0.95;--no-convergeforces off regardless of config.Recorded on the result (minimal, mirrors
cached)CouncilResult.converged: boolandCouncilResult.convergence_score: float | None. Actual rounds run is already observable vialen(result.rounds).Cache key
converge_thresholdis part of the debate cache key (make_key/Council._cache_key/_cached_runplumbing) so a converged run and a fixed run over otherwise-identical inputs never collide. Covered by a directmake_keyunit test and an end-to-end no-collision test.Hard constraints honored
call_modelnever-raises invariant and BYO-keys posture untouched (no key read/store/serialize;redact()path unchanged).modes.py, notcouncil.py.Tests
Offline, transport-mocked via the existing
patch_call_modelfixture. 142 → 151 (9 new):--rounds(unchanged behavior)--rounds, no false stopconverge_threshold=Nonedefers to config_round_convergencescorer unit testLocal:
151 passed,ruff check .clean,ruff format --check .clean.🤖 Generated with Claude Code